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^ ABSTRACT 
This paper rtports the results obtained with a group of 24 14-year-old students whan 

s. 

presented with a sat of algebra tasks by tha Loeds Modal ling System, LMS. The an sane 
students were given a comparable paper-and-peneii test and detailed Interviews some 4 
months later. The latter studies uncovered several kinds of student mlsundorstandlngs 
that LMS had not detected. Some students had profound misunderstandings of algebraic 
notation: others used strategies 'such as substituting numbers for variables until the 

f 

aquation balanced. Additionally. 1t appears that tha student errora fall Into several 
distinct classes: namely, manipulative, parsing, .clerlcal-and "random". 

LMS and Its r< 'a database have been enhanced as the result of this experiment, and so 
LMS Is now able to diagnose the majority of the errors encountered 1n this experiment. 

finally, the paper gives- a process-orientated explanation for student errors, and 
re-examines related work in cognitive modelling 1n the light of the types of student 
errors reported In this experiment. Hlsgeneral Ixatlon is a mechanism suggested to explain 
some of the mat-rules noted 1n this study. 
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1. INTRODUCTION 

The impetus for work in Intelligent CAI has two major sources: firstly, the 
practical aim of producing teaching system which are truly adaptive to the needs of the 
student and secondly the "theoretical" Interest Involvod in formulating these activities 
as algorithms. It has been argued by . Ilartloy and Sleeman 1073 that an Intelligent 
teaching system requires access to: knowlodgo of the task domain; a model of the 
student's behaviour; a list of possible teaching operations; and means-ends guidance 
rules which relate teaching decisions to conditions in the student model. 

During the last decade a number of systems have been Implemented which include some 
or all of those databases. In particular during the last 10 years a number of systems 
havo been Implemented which attempt to provide eimnnrnw Earning environments intended 
to facilitate learning-by-doing. These systems include SOPHIE {Brown. Burton & de Kleer 
19B2). GUI00N (Cloncey 1982), WEST (Burton & Brown 1982). WUHPUS (Goldstein 1982). and 
PSM-MMH (Sleeman & Hondley 1902); such systems have been called Coaches or Problem 
Solving Monitors. In this paper, we address a particular « sp ect of the problem of 
inferring a model from the student's behaviour on a sot of tasks. [I]. We shall outline 
the results of a recent experiment with 24 14-year-old students who were considered, to be 
of average ability. The issue to be considered In this paper Is- whether the models 
Inferred by the. Le«ds Modelling System. LMS. can be given a cognitive interpretation, and 
whether it 1s possiole to say something about the nature of the fluacfiiifll used by a 
student given tho model inferred by IMS. 
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1.1 The Lauds Modelling System, LMS., 

In common with BUGGY (Brown 4 Burton. 1970) LMS uses ? generative mechanism to create 
models from a sot or primitive components. Without a generative facility, the ability of 
a system to model complex errorful behaviour is severely limited. However, the use of 
such a mechanism also causes difficulties, sine* such an algorithm can readily lead to a 
comoinatorla' explosion. For example, if there are N primitive rules in e domain whore 

J. For a more detailed d1scuss»on of this, and related issues soe tho Introductory assay to 
I ntal l i u atit Tutor Ing SyiUms, (Sleeman a Brown 1902). 
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the rule-order 1, significant, thon there aro N factorial, Nl, models to, bo considered. 
BUGGY and LMS are similar m that BUGGY uses a collection of primitive bugs from which to 
generate models, while LMS uses mel-rules. incorrect rules, observed In the analysis of 
earlier protocols. On the other hand, whereas BUGGY uses heuristics to limit the site of 
Us model space, a major feature cf the US work has been the formulation of the search to 
focus each task-set on particular rule(s). [2]. As has been demonstrated (Sleomen t smith 
1981. and more particularly by Sleeman 1983 a) this technique drastically reduces the 
number of models to be considered at each sta'go. [3]. before considering the results of 
this experiment, we briefly review the production system representation which has been 
used for student models and explain the main features of the production system Interpreter 
used to execute these models, 

figure la gives a set of Production Rules, used with LMS. which are sufficient to 
solve linear algebraic equetlons of one variable. Mgure lb gives a set of mal-rules for 
this domain which have been observed in protocols analysed earlier. A task-set Is a sot 
of 6-? tasks thlch hi ghlights the use of one or more domain-rules; Figure 2 gives a 
typical task for each of this domain's task-sets and the rules which each set focusses on. 
Further, Figure 2 shows the exact format of tasks presented by LMS : this format has also 
been used In ell subsequont Interactions with the students. 

[Figures 1 and 2 about here] 

In this work, a modol is an fljtd&Lftd list of rules. Order Is significant, as the 
Interpreter executes the action of the Oiit rule in the model whose conditions are 
satisfied by the stete (I.e.. the task or the partially solved task). In this way we aro 
able to capture oracsflf i nea which is important In this subject domain. The match-execute 
cycle continues until no further rules fire. Figure lc shows pairs of correct and "buggy- 
models executing typlcsl tasks. LMS Infers a model for each task which the student works, 
producing s.ufflmacy model(s) for each Usk-set. If the student's behaviour 1s random or 
conforms to a previously unencountered mal-rule then LMS returns a null model (soo Sleeman 
1982 for more details). LMS presents tasks to a student until its example bank 1s 

2. Examples of task-sets ere given In Figure 2., 

L !hl l1 ? ny : "° "^V"' 1 "sumption that the domain was hiararchl^al'ilnd'so'wr have referred 
to the stages as lc^eUt thus modelling proceeds by first considering level 1. then 2. etc. 



ERIC 



5 

BEST COPY 



Pago 6 

exhausted or until tho student opts to "retire". 

The 1980 Experiment 

In 1980 an experiment was run with a flr oup of 15-year-old students and a yic* close 
agreement was achieved between LMS's diagnosis and those made by a group of investigators 
who gave the students individual intorviows on analogous tasks (Sloeman 1982). in one 
important respect LMS and the investigators differed. The design of LHS was such that if 
the student did not make ah error with say XTOLHS when it was introduced, then LMS assumed 
XfOLHS would be used successfully at an subsequent levels. This experiment showed that 
this was not a valid assumption. For example, some students who were able to correctly 
work tasks of tho form: 

3 • X ■ 4 • X ♦ 9 
bad trouble on the following type of task: 

12 • X - 2 • <4 • X ♦ 6> 
where they appeared to forget to chm& the sign of the X-term when the side is changed, 
and thus we have seen 20 • x ■ 10 returned as an answer. 

It was. in fact, easy to remove this assumption from LMS's code, but unfortunately 
the modification led to an explosion in the number of models to be consi.' sd. and so a 
reformulation of the algorithm was carried out (Sleeman 1983a). 

As a result of this experiment, we believed that students' behaviour on algebra could 
be largely explained in terms of manipulative maWules. namely mal-rules in which one of 
the substeps is omitted. 



2- All OV ERV I EW al EARLIER Il£UVAiJl UQfi& iu CQGHiTivg modei mjr. 

BUGGY (Brown & Burton 1978) analysed the responses which students gave to 
multi-column subtraction tasks. The system reported a diagnosis for each student In terms 
of correct procedures, or procedure: which had some of their substeps replaced by 
incorrect variants, called bugs, 
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Young 4 O'Shea 1981 point out that although BUGGY produces models that behave 
functionally as the students, those models are not very convincing as psychological 
madels. Many of the bugs appear to be very similar (many are connected with borrowing 
from lero) yet this relationship is not made clear. More particularly, Young & O'Shea 
she- that some of the BUGGY data can be analysed more simply in terms of certain 
competences being oniUtflfl from the iueal model. 

» 

Repair theory (Bro.n & VanLehn 1000) is a further attempt to provide a psychological 
explanation for the same data. Here Brown & vanLehn take a correct procedure for 
performing subtraction and apply a deletion operator to the procedure. This perturbed 
procedure is then used to solve tasks, when it encounters an impasse, such as a situation 
where it is about to violate a precondition (e.g.. attempting to take a number from 0). a 
repair is applied to the Perturbed procedure, and it attempts to continue solving the 
task. This process also uses critics to throw out somo repairs which are considered 
impossible at a given Impasse. 

More recently VanLehn ig 8 3a has suggested a variant of repair theory, which does not 
delete .tops from procedures - as it is argued that the blocking, or inhibition, of the 
deletion operator was unprincipled. Secondly this version overcomes the difficulty that 
certain core procedures CMOfll be generated easily by rule deletion. Instead. VanLehn has 
suggested a series of core procedures, which correspond to the various stages of 
instruction (c.f.. Sleeman & Smith 1031). From this perspective an impasse occurs when 
the student encounters a sub-task which he has not learnt, or has forgotten. 

Both variants of repair theory explain what Brown and Ms coworkers have called bug 
migration, namely that with the same t iP e of task, the student may display different bugs 
both during tho same test-period and between different tests. Moreover. VanLehn 1901 has 
analysed protocols in which It was possible to generate all the bugs in an observed . 
migration class by applying different repairs to a common (paitieliy learnt) core 
procedure. So VanLehn suggests consistent bugs can bo explained by supposing tha student 
stores the "patch" and merely uses it with the neat tank. The explanation for bug 
migration is that the patch is OJU retained and tha. one of the repairs is selected 
randomly. 
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The Illinois group (Davi*. Jockuseh ft McKnlght 1078) ha raportad algebra students 
ov.rganar.llslng from Instances, using an -old" oper , . .aad of a mora r.caniiy 
introduced one £4], and regressing under cognitive load. M*w 1082 has further analysed 
those students' performances and has suggested a number of high-level schema «h1ch explain 
series of observed errors. These include her -extrapolation principle" which explains why 
a student who has seen the legal transformation: 

(A • 8) a C A a C • 8 a C 

would then write: 

(A ♦ B) a C «•> A a C + 8 A C 

She also discusses the confusion which seems to arise between the notations of arithmetic 
and algebra. For instance, she argues that as 3 3/4 is to be interpreted as 3 ♦ 3/4 It Is 
not unreasonable that the student should Interpret the algebraic expression. 3X as 3 ♦ K. 
C5J. [6], 



4. ♦ instead of •, • instead of Exponential 



6. Although this explanation wo-jld explain aoJDfl of our observations «t..rilnt. 4 no , 



see 



6. Similarly, our earlier work provided an additional data point for the"l9fll «xnerWt a. 
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3. ME 1901 EXPERIMENT WITH LMS 

The 1081 experiment was carried out with the tauami modeller, LMS-I1. [7], but with 
the same data-base of rules and tasks as used In tho 1980 experiment. This fi roup of 24 
students, average age 'years 3 months, were Judged to be of average ability at 
mathematics; however the results were diamaUcalLy different from the earlier group's. 
C8]. Indeed many of their difficulties ware not diagnosed by IMS-II and had to be 
analysed by tho investigator. This analysis was made very difficult because it had been 
assumed that students would at most make one or .,o minor manipulative errors, e.g.. 
changing side and not sign, and so LMS had been designed to allow the student to Input His 
or her final answer, and as many intermediary steps as he chose. in Figure 3 we give 
sample summaries produced by LHS-II for students' online interaction, together with the 
mal-rules which the iflJUiliaUot suggested were eppropriete for each task-set. In Figure 
4. we summarise the complete set of new mal-rules which the Investigator considered 
explained the students' behaviour with LMS-II. 

Note that by stating that a protocol can be explained by a mal-rule. say. of the form 

M • X •> rt ♦ X 

(Figure 3a). we do not wish to Imply that given a problem of the type 

3«X + 4«X-6 

that the student would produce tho response; 

3 ♦ X ♦ 4 ♦ X ■ 6 
Indeed, we have seen several students write 

X ♦ X - 6 - 3 - 4 

and when asked to provide Intermediary steps they have said categorically that there were 
none as the above was done in "one step". Nevertheless we are happy to accept that both 
forms are explained b y the mal-rule; the first form however requires that several 
additional rules are executed in order to get it into the sta o given by the "second" 

7. Revised to cmant the assumption that if an error 1s"iormade"iith"a"uirit"the stage it 1s 
Action ? ' * 41 ^ StU<,ant *" } U " thB ru,e corroct1 * on •» "J!i!u!"t !«.i;K.! 1 ..! 



<; h n?!?! 1 °I !? 9Sa $ fl! de !!^, had baen 1ntrociu «1 to algebra several yc.rs earlier in their middle 
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student. (It should be noted that the mal-rulos given in Figures 3d. and 3e are more 
comprehensive and carry out several housekeeping steps. The differences between basic and 
comprehensive mal-rules atft significant when ono tries to perform remodlal Instruction, as 
It Is Important to ensure that the grain of the Instruction. matches the student'*.) 

[Figures 3 and 4 about here] 
As a result of analysing those summaries, a number of questions were raised: 

-What Is the crucial difference between the task-sots which a particular student Is 

end Is not able to correctly solve? 

- Ooes the student's. perception of algebraic tasks vary from one task-type to 
another? • 

Unfortunately, as the school vacation Intervened. It was not possible to meet with the 
students again until September (1981). Oecause of the time that had elapsed, the students were 
given a papor-and-pencll test which covered comparable tasks to those used by IMS. These touts 
were analysed In detail by the Investigator, and a* a result of this certain students were 
given detailed diagnostic Interviews. The neat sections give more details of these steges. 



IhO. SQPtemtLflC Pau er-anri-P«nri1 XuAt 

From a comparative review of the May and September data, see Sleeman 1983c for the 
details, we concluded: 



1. The performance was generally considerably better in September than 1n May. (Note na 
additional teaching In Algebra had boon given, however the students had presumably 
done some self-study 1n preparation for their Juno examinations.) 

2. A considerable number of tasks were not solved on the written test (whereas IMS 

* 

insisted the student gave a i sponse to each question). 

3. Some students who appeared to have "wild- rules on particular tasks in May. seemed to 
solve this type of task correctly In September, e.g., A85. 
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4. Soma students whoso behaviour had been "random" or "wild" In May had now settled to 
use mal-rules consistently, e.g., student ABU. 

8. One student. >AB7. gave multiple values in an equation where X occurred more than once. 

6. Many of the students made the common precedence error, namely given a tat ft of the 
form: 

2 ♦ 3 • X • 11 they return 6 • X ■ 11, 

, As e result of this comparison It was decided to interview all those who appeared on the 
written test still to hpvu major difficultly, but not to Interview those who hod anl* common 
"precedence" errors, or those who hod had major difficulties which appeared to have -cleared 

up". 



The Interviews proved to be remarkably revealing as the students without exception wore 
extremely articulate. These dialogues were recorded, fibres 6&6 have been reconstructed from 
tho tapes and the worksheets. 

The investigator presented the student with a series of tasks and asked him. or her. to 
work e.ch one explaining as he went along exactly what he was doing. In some cases the 
investigator asked the studmt to tell him which of two alternative forms were correct ana 
frequently asked the student to explain why. The tasks presented were different for each 
student, and were based on the difficulties noted 1n the Individual's September test. The 
interviewer thus started each session with a list of task-types to be explored, but often 
generated particular tasks as a result of answers given to the planned tasks. 

The following Is a summary of the main features noted during the Interviews: 

1. Some students "searched" for solutions (i.e.. tried different values for X). (Section 
3.2.1). 
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2. On« student computed a separate value for OAiih X given In the equation. (Section 
3. 2. 2). 

3. One student maintained that there were a number of quite distinct ways of sowing an 
equation; even when It is demonstrated that each approach led to different answers. 
(Sect<«n 3.2.3). 

4. Some students have ,, hard , . , .. consistent, mal-rules. (Socton 3.2.4). 

5. Some students have the correct rules and can explain why 1t is not permissible to 
perform the Illegal transformation, including the illegal transformations that the 
student appeared to use in May. (Section 3.2.6). 

Each of these points is discussed in the following sub-sections. 



3.2.1 Searching £££ S olution^ 

Searching for a solution appears to be a vary common way of solving equations with 
students beginning algebra, and presumably arises because the Initial equations presented c ould 
be solved in this way. When givon an equation of the form: 

3 • X ♦ 2 - 14 

such students substitute X-l, then X-2, then X-3.... until the equation balances. (See 
Sleeroan 1903c for further ddftalls of student ABU's protocol). [9]. 

Further, student AB11 solved tasks of the form: 

3 • X • 2 

as X ■ -i, explaining she suJaoiCled 3 Trom both sides. 

It is Indeed intriguing to watch students changing their approach when solving tasks of 
lh<i latter form depending on whether the task is solvable by "search". Students do not appear 



9. Indeed, in a more recent test with 100 13-year-olds, it appears that about 95X of them use 
this approach. Intelligent Tutoring Systems and loach rs should suspect that a student Is 
using a naive algorithm if he appears to be unable to solve tasks where the variable is a 
negative Integer, large-integer or non-Integer. The teacher should be concerned because the 
naive aUiorlth- is only applicable to a sub-set or algebraic equations, and hence should be 
deemed a significant weakness, and one to be remedied. It seems clear that the use of 
simplistic tasks leads to a naive algorithm which causes major conceptual difficulties on more 
advanced tasks. 
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to believe u»at all equations of the same form should bo solved in the same way. (Clear Jy this 
point should have boon discussed In an interview with those students.) Such students are often 
unable to solve correctly equations which contain multiple Xs; they attempt to » value, 
for all the Xs in the equation, sea the next subsection. 



3.2.2 tiiitlala tat i 

In this subsection, we report a student who has a very strange, but nevertheless very 
CiUU^iaiLt algorithm, for solv1n 0 tasks involving 2 xs. When student AB7 was originally 
working at the terminal, sho was heard to mutter: 

"If this X was 2, then it would work if this second X was 4". 

Not only was this .tudent consistent In both the papar-and-poncll exercise and In the 
Interview, she was able to explain what she was doing. Given the task: 

3 • X ♦ 2 • X ■ 12 

She gave »he following explanation; 

"What I o is take the 3 and I make the first . <ial to 2. so I write: 

3*2" 

When asked by the Interviewer why the -first" X Is equal to 2. she explains that It's the next 
number along, and then added "I think this is the wrong thing to do. but that's what I do". 

She thon continued "... and then I write down the + 2 making 

3 • 2 ♦ 2 

I then work this out. this is equal to 8 and so the second X Is 

12 - 8, that 1s 4". 

•She then completed the solution and gave the 2 values for X. and so the final state of her 

worksheet was; 

3*2*2*4-12 
X • 2 
X • 4 
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She used this algorithm consistently on 9 tasks, see SI ••man 1083c, [10]. 



3.2.3 AUarnaHu* BifttttUUffll 

Although student A017 was able to solve several task-typos correctly, he was easily 
"distracted" and quite unable to tell the Invostlgator why the Investigator's "alternatives" 
*ere Illegal. On tome tasks the student suggested iftjutU Illegal solutions, and again was 
really unable to distinguish be two en them. (See Figure 5 for details). On the other hand, 
this student did give as an aside a rationale for his "method", namely "collecting all the Xs 
to the LliS and all the numbers to the RMS", which will be discussed In Section 5.1,2a. " 

[Figure 6 about here] 

3.2.4 "HarH"/rnn««^f nn | | Hal-Wulwa , 

Hany of the students used consistent mal-rules. Just over half of the 24 students we sew 
mis-handled precedence In equations of the form: 

2 ♦ 3 • x • g 

Part of a protocol for one such student Is given In Figure 6.1. [11]. Figure 6. II is part of 
the protocol produced by the student discussed in Section 3.2.3. where he consistently epplles 



; a VhA 1 y : 7! $uppo " d t " 1 » to a very idiosyncratic algorithm, but subsequently 
noted that a variant was used extensively by 13-year-olds. For example we have seen: " c,ueniiy 

3 • X ♦ 4 • X ■ 3 

"solved" as: 

. 3 ' 1 ♦ **0 • 3. making X ■ 1 rnd X ■ 0. 

Similarly. 

3 • X ♦ 4 • X - S3 

has been "solved" as: 

3*22 ♦ 4*8 ■ 60 ♦ 32 « SO. 
Moreover, in ••complicated" cases the two sides often are not "balanced". Thus we have 

sten 

3 • X ♦ 4 • X - 100 

"solved" .tsi 

3*30 ♦ 4«2 ■ 100 

and when asked the student explained that "this one did not work out exactly". 

Note that these students frequently solve tasks of the form: 

3 • X ♦ 4 ■ 10 

by "search". 

Jim^uu? !! k haW ?K d ;| COtf<Jr8d l " at 9 <>* of » samp1e""of"I5"^ar"IoloS had precedence 
difficulties with arUhmiUk expressions involving " ► " and "•" operators. 
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a further Intriguing transformation to a complete sot of tasks. In order to understand this 
protocol fully we have suggested that a nttimUZaliaa step takes place between stages 1 and 2 
of say protocol a). That 1s, we suggest that when the student applies the mal-rule to the 
original task, this results In an "unusual" form which the student then "normal laes" before 
continuing to process the rest of the task.. (See Sleeman 1383c for a lengthier discussion of 
"normal liat Ion"). 

[Figure 6 about here] 

Student AB1S, Figure 6. III. Is remarkably consistent with his mal-rules over a whole range 
of Usk-typos. Note the application of his algorithm to task c) which Involves 3 x-terms. (To 
give him Justice, he realises that he had got tasks d) - g) wrong as ha noticed that the 
equations did not balance whan ho substituted his answers back In). Further, having worked 
task h). he noticed that when ho moved the 4 across to the right hand side, he changed the 
sign. So he suggested that when he moved the X (associated with 2 • X) to the LHS, he should 
also change Its sign. He said; 

"X - X Is 0, and so the LHS became 0 and the RHS did not" 
and so realised that this proposed solution was Impossible However, for good measure ho also 
worked task 1) with the "revised" algorithm. 

In the course of our discussion, this student also gavo the basis for his "algorithm" 
which Is discussed In more detail In Section 5.1.2a. 
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3.2.5 "Sajted Simla * 

In September student AU5 worked correctly tasks which she had got cons Unfitly wrong In 
May. namely task-sets 7 and 8. For task-set 8 she appeared to use .rul-rule; 

M • X ■ N • X ♦ P X ♦ X ■ N + N ♦ P. 
Moreover, when presented with a fallacious alternative during tho September interview, she was 
able to spot it and to say aJiy It was wrong. For e«ample, "not able to add a number to an X 
term", "not able to separate a number from an X term", etc. (see Sleeman 1983c for more 
details). In May. this student showed o lack of understanding of basic algebraic notation 
which appeared to be remedied by September. To see whether this was the case the investigator 
also presented tasks from sets 12 and 13 of Figure 2, i.e.. tasks of the form; 
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m ► n • x ♦ p • x • ; and m • x • n ♦ p»<q • x ♦ r> 



AH of which sht worked correctly and was able to verbalise tha stages sha want thrown. An 
aquation which contained an -unusual- variable. AA, was also prasentad and again this was 
worked correct!*. Similarly aeveral other students showed substantial -progress", and again 
this was associated with the ability to Aiflloia what they were doing. 



4. SUBSEQUENT UPGRADE of IMS and Its DATABASE 

The set of rules and mal-rules used 1n the 1901 experiment has subsequently been enhanced 
to include tha additional manipulative- and parsing- mal-rules confirmed by tha student 
interviews. [12], Additionally, the code of IMS has been extended to deal with mal-rules which 
have a somewhat different character from manipulative rules. The extended IMS with the 
enhanced database is ebla to diagnose the majority of the errors encountered ' both In the 
on-line sessions carried out in May 1991 (sea Figure 3) and In the Interviews (see Figures 546) 
Sleeman 1983c. 
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S. COGNITIVE MODELLING and an INTERPRETATION of the RESULTS of the 1981 EXPERIMENT. 

There is a stoadily growing body of data about how school and college students solve 
algebra tasks. Paige & Simon I960. Lewis 1900. Davis. Jockusch 4 Mcknight 1978. Kuachemann 
1981. Sleeman 1982 and Sleeman 1983c. The major thrust of this paper Is the analysis of the 
experimental results reported here In terms of related work In cognitive modelling; Section 2 
givos a summary of this earlier work. 

« 

tfl f U non t OhxnrwAHrti^ ton Expnr tinum 

1. Students appear to regress under cognitive load, see Sleeman 1903b for details. 

12. Some of the mal-rules noted In Figure 4 were not seon 1 Tine "interviews I now h«n. u - 
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2. There appears to be a number of. clearly Identifiable imi of errors. (Section 6.1). 

3. Students use a number of alternative "methods- to sol*, tasks* of the same set. 
(Section 6.2). 



JL1 Class 1 f leat-ton q± QtjiQrVftl 4iuHa.lt, tt££fl£& 

We propose a classification of students' errors observed In this experiment, namely, 
manipulative. parsing, clerical and "random". Practically this classification 1s of 
considerable Importance as It enables one to Qlve aflflronrlntn remedial Instruction for the 
several types of error. In case of "manipulative" mal-rulas it would appear that the student 
basically -knows" the rule, but due to cognitive overloed or Inattention. 1s omitting 
substep(s). The parsing errors appear to arise from a profound ml&Uitdar&Unfllflg of algebraic 
notation, and so have to be remediated itftty. differently. Additionally. In this section wo 
suggest mechanisms for several of these error-types. 

4*1*1 Manipulate Vtt E rrors 

We define a manipulative mal-rule to b* a variant on a correct rule which has one 
sub-stage either omitted or replaced by an Inappropriate or incorrect operation, c.f.. Young & 
O'Shea 1981. For example. MNTORHS 1s a mal-rule which captures the movement of a number to the 
other side of an equation whore the student omits to chtum the sign of the number. We suggest 
that all tho mal-rules reported in Figure lb and those numbered 7-9 1n Figure 4 fall within 
this category. Also In Sub-section 6.1.lb we briefly discuss th. (apparently) related 
phenomena of confusion of operands. 
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al A ixtiaoa Lac paftBrd ting mfla.lpulii.tjta (Mirailoi 

In Figure 4 wo report 3 new (manipulative) mal-rules, variants of SOLVE, SIMPLIFY and BRA3 
respectively. [13J, which can be explained by a deletion mechanism, as can the "original" 
mal-rules given 1n Figure lb. Not a that this schema would ALSO generate many mat-rules which 
w b have not yet observed. In the next paragraph we suggest why some of the possible mal-rules 
sre not observed. 

a wxcJaax aa SOUK. The student realises he has a task In which the SOLVE rule should be 
activated and forgets to apply one of the operations, namoly dividing by M. SOLVE has thro* 
principal actions: noting down N. the divide symbol and M. and so this mal-rule could be said 
to be omitting some of the principal stops. It appears that students have an Idea ebout the 
acceptable FORM of answers and so given: 

M • X ■ N. they do NOT produce X • 7M or X - N7 

a . v a ri a nt aa SI MPLIFY . Exsmples of the two rules given here, which have occurred reasonably 
frequently are: 

X • 6/4 -> X • 3/4 
X • 6/4 ■> X » 6/2 

Again we argue that the above observations can bo explained If we assume thet this rule 
has several pr1nc<o*l steps Including calculate the common factor, divide the "top" by the 
common factor, ano divide the "bottom" by the common factor. Eech of these mal-rules 
corresponds to one of the letter steps being omitted. 

A Uiiijini oa BJ1A2. We have seen the task: 

6»X-4»<2»X*3> -> G»X»4»X*12 
BRA2 is a more complex rule with several steps end so one might expect to find a 



U. The variant on SOLVE reported 1n Figure 4 Is: 

H • X ■ N •> X-M 

Two variants on SIMPLIFY reported in Figure 4 ere: 

M * X - N •> X ■ (N/F)/M 

H • X - N •> X - N/(M/F) 

where F Is e factor of M and N. 

The variant on BRA2 reported in Figure 4 Is: 

M«<M • X ♦ P> ■> M • X ♦ M*P 
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correspondingly larger number of mal-rules. This Is indeed true. This "new" mal-rule also 
conforms to the pattern noted above, as it can also be explained by the omission of one 
sub-action. 

ill COAlUi.ioa Ol Operand*. 

We have noted errors of the following form: 

5 • X ■ 12 ■■> X • I 2/12 
whore clearly one operand 1* confused for another. Norman 19fll explains Such slips by saying 
that thoy are a consequence of a noisy processor. 

cl "Grain alia" ami oaqJauIaLLua mIlluJju. 

There Is a very real sense in which detailed analyses cf manipulative mal-rules allows one to 
infer the substeps processed by students, and this In turn allows one to predict the set of 
mal-rules that will be encountered In a domain. (Bearing in mind the Idea of acceptable form 
outlined above). Further, one might argue that the representation of the tasks should be at 
this "lower- level t the Justification for the representation chosen (see Figures la&b) Is that 
this eppears to be more consistent with the collected verbal and written protocols for students 
solving these tasks. 

The schema discussed sbove for generating manipulative mal-rules by omitting, or 
modifying, one substep Is thus consistent with Young end O'Shea's modelling of subtraction. 
Further, we believe that confusion of operands can bo seen as a variant of this same mechanism. 

Incorrect RflacaiQnla.ti&c, cl ma Task or. Eaula Etcun . 
We have c.uegori*ed the first 6 sets of Mal-Kulos in Figuro 4 as ones which summarize what 
happens when a student "mis-sees", or mls-parsos, an algebraic equation. Wo assert that many 
of the students whom we Interviewed carried out steps of the computations In ways which would 
not fall within the definition glvoit earHer for manipulative mal-rules. relow. wo give 
typical protocols for two students working the task G«X • 3»X ♦ 12; 

6«X - 3«X ♦ 12 6M • 3«X ♦ 12 

9»X • 12 X + X ■ 12 ♦ 3 - 6 

X ■ 12/9 2«x • 9 

X =» 4/3 X ■ 9'2 
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When we prassad the "first" studant for an explanation of ho- the original equation .as 
transformed Into the second, I.e.. g* X . 12 , the student talked about moving the 3»X term 
across to the left hand side. Thus the interviewer concluded the student was using a variant 
of the correct rule, namely a manipulative mal-rule. >ihen the "second" student was pressed he 
simply asserted that the change from the original equation to the second form "was all done In 
one step". Hence the Intorvlewer concluded it was a very different iyjm of mal-rule Involved 
and ool a simple variant on the correct rule. Thus tho Interviews provided aas.nHei 
additional Information as, of course, the second student's protocol could be explained by the 
use of HXTOLHS and the mal-rule: 

H • X -> H ♦ X 

which some people might wish to argue constitutes a Banlpulat.lxft mal-rule (replacing the • 
operator by the ♦ operator). This Investigator maintains that such a transformation reveals a 
profound misunderstanding of algebraic notation and so UmlA be considered as a mcjJa, 
mal-rule. 

Additional "evidence" for the distinction between manipulative and parsing mal-rules comes 
from an understanding of the likely representation of the equation for the two groups of 
students. Figure 7a gives a correct parse tree for the equation discussed above. Highly 
probable Inadequate representations for the equation, which are consistent with observed 
mal-rules. are the linear algebraic string. I.e., the usual written form of an algebraic 
aquation, and the not-so-deeply nested tree given in Figure 7b. These latter representations 
suggest that the student has failed to appreciate the semantics of algebraic expressions - and 
sees the solution of algebraic equations as a symbol manipulating task. We collected 
considerable evidence to support this view earlier, Sleeman 1982. and in the 1961 experiment 
(se-j figure 6.11a), where a student transformed: 

2 • X + 4 • X « 12 into • X • X - 12 - 2 - 4. 

*i Stilfliua Iflf. "generaling" Parsin g m al-rule s,. 

In the course of the Interview student AB18 explained that he was carrying out the 
teacher-given algorithm of: -Collecting all the Xs on the left hand side and collecting all 
the numbers on the right hand side", end added that he was not really sure what to do about the 
"extra multiply signs". Student AB17 gave a similar explanation for his actions. This gives 
us a iitLema for generating mal-rules. In this section we explore this topic further. 
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For example given the task-type: 

H • X ♦ N • X - P 
This schema glvaa the following "action aides" for mal-rules: 

X ♦ X • P - M - H 
X*X-PfM*N 

where In the second case the X coefficients are treated "specially-, i.e.. the coefficients of 
the Xs were taken across to the RHS of the equation but the signs were NOT changed. 
Additionally, there Is the form given by student AB17. and quoted 1n Figure O.II, namely: 

•X'X-P-M-N 
which he went on to "normalize" (see Section 3.2.4) to: 

X • X ■ P - M - N 

and Us "complementary" form: s " y 

X«X-PfM*N 

Similarly, given the task-type: 

M«X-N»X*P 
this schema creates the following forms: 
X ■ N ♦ 9 * N 
X ■ H ♦ P ♦ M 
X*X-*eP-M 
X*X-N*P-M 
X - X - N ♦ »> - M 
X-X"«eP*M 

For example on task h), student AB18 suggested the use of both the third and the firth 
forms (see Figure 0.2II). 

As argued above, unlike the manipulative mal-rules. the oaring maWules cannfll be 
explained by omitting a component. Neither does it seem that they can be explai^d by 
performing a repair to a core proceduro. unlen one is prepared to broaden one's view of a 
repair to include the schema which were observed with students A817 and AB18. and the 
^extrapolation" procodure noted by Ma v z. [14]. 
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mechanism discussed in Section 5.2. a 
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fi.^lU Clerical tiuu. 

Analysing som of the protocol*, on, U Happy wU l> the explanation that some "slips- occur. 
For example: 

10 • X - 26 •> X ■ 25/18 
2 • X • 6*6 -> X ■ 18 

In tha first casa the student has probably soon the "0" as an -8". In the second he has 
probably made en arithmetic error, [16]. DEBUGGY (Burton 1982) considers an answer to be . 
-number-oond sup- if the answar is within 2 of the cor oct one. The second , Hp fl 1vn abova 
could be explained If *. had an analogous algorithm for tha evaluation of multiplicative 
expressions. However, the first slip, a "visual" one, clearly could not be. So we suspect 
that to account for the .variety of "slips- encountured In this domain a more sophisticated 
approach, c.f.. that advocated by Norman 1981 would be necessary. However, m have not thought 
this worth investigating as clerical errors have so far been relatively infrequent. 

ftJLJi IW gfl VUnft&pJL&lMfl Error* . 
Many of the mistake not explained so far may ba due to the consistent use of mal-rula* which 
so far we have aaX Identified, [18], 



!uZ U&iflfl AUsmfltlVfl MfllMdi or Bug Migrating 



Repair thaory givas a neat explanation for the observed phenomena of bug migration 1n the 
domain or multi-column arithmetic. Brown 8 Vantohn 1980. namely that tha student will use a 

rolatad family of mal-rqles. and possibly tho f^rjiaa rule, during . single session with a 
particular task-sat. 



15. Given the earlier definition of manipulative mal-rules. it aooears th.t th. 
reported above have a different nature At the very east lf^Kv M it*.**/.! «5 
manipulative mal-rules. they are m algnj^ manlputatlve^ai^ulosf 1 1nSt ' nC " of 



do^icJ 5 ti h issi 1 Jt C Jhr!JIve s n^t^ Ofl,an i!' n0 I"" 1 th0r ° is * nood 10 ^*™<* *™ computational 
TZll ! i <nvestiqator. A preliminary system hus been implemented which has alroadv 

given several "explanations" not spotted by the Investigator. a'roaoy 
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There seems to be An alternative explanation which should also be considered. Although a 
task-set may have been designed to highlight one particular feature, the student may apot 
completely different feature(s) and these nay dominate his solution. £173. Repair theory 
accounts for some bugs by hypothesizing that the student had not encountered the appropriate 
teaching necessary to perform the task. Suppose wo make the converse assumption, that the 
appropriate teaching had been carried out. and further suppose thwt soma students gain 
competenco in this domain not by boing laid the rulos but rather by iflftuxtog rules for 
themselves, by noting the transformations which are appl led to tasks by the teacher and In 
texts. [18], [19]. It seems reasonable that the student's Inference procedure should be guided 
by his previous knowledge of the domain. In this case the numjer system, and that the student 
normally Infers several rulos which are consistent with the example, and not Just the correct 
rule. Indeed due to some missing knowledge the correct rule may not be Inferred. (And so the 
fact that the student na^ar us the correct method along with several buggy methods Is oal 
evidence that he has HOT encountered the material before.) We shall refer to this process as 
Knowledge Directed Inference of rules, or mls-goneral iiation for short. 

Suppose, the student saw the following stages In an algebraic slmpllf icatlonj 

3 • X • 6 --> x • 6/3 

Then he might Infer 

X " RHS number /IHS number OR X ■ LARGER number /SMALLER number 

Further, we suggest that schema such as that erticulated by students AB17 and AB1S could 
have been inferred by the process of mis-goneral Nation. 



17. Earlier Sleeman and Drown 1982 have argued: - Perhaps more immediately. It suggests 

that a Coach must pay attention to the sequence of worked examples and encounteredt.sk 
states from which the student 1, apt to abstract (Invent) function Jnv.r 
. 8 !:i U ^i! a L M ,?? U : P car ^ un > « instructional designer plans I soquenc . 0 sample 
he can .over know all the intermediate steps and abstracted structures that a student wll 
flonarate while solving an exercise. Indeed, the student may well produce Illegal steps In Ms 
to ut.on and from these invent illegal (algebraic) "principles". Implementing \ system w"! 
coun ty " ,0ph1St ' Cat<0n P^entw major challenge to the ITS/Cognitive Science 
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18 Note I am nai claiming that there is a single, mechanism. Matz has provided another 
mecnanls'/i namely, that some students use an -extrapolation principle" to extond a method they 
know woiks in one content to further analogous contexts. * 

19 Independently. VanLehn 1083b has como to a similar conc1usion""the"sie^a"system described 
In his thusis rellos heavily on Inforonce. 
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We will surmise how a student would use such a rule-set or schema, we will suppose that 
the abler students actively experiment with different "methods*, and use their own earlier 
examples, examples worked by the teacher or in the text to provide discriminatory feedback, 
from our experiment with 14-year-old students we have direct evidence that soma students, e.g.. 
students A817 and ABIJJ. are aware of having a range of applicable rules and being unsure of 
when to select a particular method. That study did not provide any insights into the 
rule-selection processes used by those students. We could suggest the common default. I.e.. 
that the process is random. However, studies in cognitive modelling have already discredited 
this explanation many times, so we will . postulate that ' the process is deterministic but 
. currently -undetermined". It is further suggested that tasks whlc.fi show a rule is Inadequate 
will weaken belief in the rule, but once a (mal)-rule is created It may not be completely 
. eliminated, particularly if tho "counter-examples" are not presented to the student for some 
period. Thus given this view point, the phenomena of bug-migration occurs because the student 
has Inferred a whole range of rules and selects e rule using a "black-box" process. Given a 
further task, he again chooses a method and hence selects the,sr<ie or an alternative algorithm. 
Influenced partly by the relative strengths of the rules. That is. If the relative weights are 
comparable it is more likely that the student will select a different method for eech task. If 
one weight dominates then It Is likely that the corresponding method will be selected 
frequently. Further, if only one (mal) rule Is generated by the induction process then this 
approach predicts that the student will consistently use that rule. 

We suggest that many of tho bugs encountered in the subtraction domain can be accounted 
for by this (Inference) mechanism. For Instance the Smal ler-rrom-Larger bug where the smaller 
number is subtracted from the larger iadjmfljtilaAt of whether the larger number i 5 on top or the 
bottom row seems one such example, Brown 4 Durto.< 1970 and Young i O'Shea 1981. Brown & 
VimLehn 1980 report that because borrowing was introduced, with one grc-jp of students, using 
only i -column tasks these students Inferred that whanever borrowing was involved they should 
boirow from the loft-most column, their "Always-Borrow-Lef t" bug. (So it appears important to 
ensure that the example set Includes some examples to counter previously experienced mal-rules. 
Indeed it seems as if task-sets can be damaging if they are too preprocessed and contain too 
little "intellectual ruffage"; Hichoner 1978 makes a similar argument.) Additionally. Ginsburg 
197/. quotes sevoral instances of young children Inferring the name "threo-ty" for 30. given 
the names for "40". '60". "00" and "3". So given the wealth of experimental evidence this 

*•"■'. '* 
• ■*• > * 

■ •% * 
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alternative explanation slioulii be glvon serious consideration, 

further, I have two philosophical reservations about repair theory. firstly, by some 
mechanism not articulated all students acquire a common set of Impasses, a „d moreover thoy 
consistently observe these. Secondly and more significantly, repair theory, which sets out., to 
explain Misit individual differences at the task level . itself proposes a specif 1c mechanism 
CQMM to all students. [20]. On the other hand, mis-gonerol imlon predicts that the 
individual's initial knowledge profoundly Influences the knowledge which Is subsequently 
Inferred. a.,d captures the sense In which learners are active theory builder, trying to find 
patterns, making sense out of observations, forming hypotheses, and testing them out. 



4*1 Conclusions, 

\ 

firstly, wo have two explanations for some of the misunderstandings noted with algebraic 
notation. Namely, that given by Mat* 1962 and that given by s tudents. ABU and AB18. see 
Section 5.1.2. Certainly. Mats'* explanation explains Mffia of our observations, but not all aa 
in some cases the coefficients are treated "specially", and their sign Is oat changed when they 
are moved to the RHS. For example, wo have observed; 

3*X*4>12»>X*12+3.4 
I.e., the student changes the sign of the 4 but Mi the 3. 

Secondly, there are two hypotheses which explain bug-migration: the one given by repair 
theory and the one put forward here, namely nis-general Nation. Of course it is possible that 
each may be applicable In different situations. 

» 

Thirdly, several "algorithms" have been presented for creating student models. 1 believe 
these are suggestive aoout the students' organisation of this knowledge and about the processes 
used when students solve (these) tasks. Repair theory suggests patches are made to Incomplete 
core-procedures. Young and O'Shea suggest that it Is adequate to'tak. a correct procedure and 
•aerely delete substeps. The data for the algebra manipulative mal-rules can be. adequately 
explained by either. However. Young and O'Shoa's approach seems inadequate to explain the 



20 " ,!??!!*' ! am """•^a" I"** '"any theories of (child) development do not accent th. 
;" !;'!:•;; b "»9 significant individual differences in de^lopmon ? buf me" y' n h! 
indivlduol s of progress and tho level of his final maturation. 1 1 
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parsing mal-rules. Indeed, we have to extend the raised repair theory before the rosults 
reported' here can be accomodated. (An analogous extension Is needed to accomodate the 
0av1s/M n u results). This paper claims that thore are two very different types of malrules at 
large with algebra students - namely manipulative and parsing mal-rules. The existence of this 
second category of algebra errors, and many of the maWules collected 1n other areas, appear 
to be best explained by. a further mechanism, namely mlsgenorallsatlon. [21]. 

fourthly, there Is evidence that once InfiTrnrt. rules are additionally WA LUul 
Incorrectly. ! suggest that the mechanlsm(s) described by repair theory. Young & O'Shea. and 
Sectlor 6.1.1 are appropriate for the appllr.ninn stage, whereas mlsganerall-ation 1s e more 
plausible mechanism to explain rule acnuiilliaa . 



nJ' lh I h ;«^n'/!J? rnpar1S ° n f, 0f 6 ^ 1anaUon * (or thoorles) are important In that they remind 
of the essentially pragmatic naturo of Cognitive Science. 



q 26 

F BEST COPY 



US 



Page 20 



6. REFERENCES 

h ^;! , M.!r 0 "\! > :"; t .!!f" t0 " ( ,9?8 >' Diagnostic Models for procedural bugs 
in basic Mathematical Skills, in Cggi!!^ SfiifiMflu. ppl66- 192. 8 

h,„. i* S ;- B T n J K : J ,y anU,,n < l900 >* Re P a1p Thaop y s A flaneritlve theory of 
bugs 1n procedure! skills. CflflOlUvA StUnxa- 4. pp 379-428. 

J.S. Brown, R.R. Burton, & J. do Kleer (1082). Pedaaoolcal Natural 

i„ a ?2???f. n ; n ! /T ,8 c dQa B "<M"«"1"fl Techniques 1 In SOPHIE I? II and III I 
JnlaU^m Tutor In o Smem. edltod by 0. Sieeman & J.S. Brown. New York- 
Academic Press, pp 227-282. new Tor*. 

A. B.untfy (1982). Personal communication. 

i I; R :-..? U C t0n ? J - s - Dro " n (»981). An Investigation of computer coaching 
i «»l earning activities. In laUUlgfiJU Tiitnrlag Sjalumk. edited by 0? 
S.eeman £ J.S. Brown. New York: Academic Press, pp 79-98. 

T n -i?;!L. BUP - t ? n S 1902 ]* 4 D1 *8 nos1n 9 1»"9S 1n a simple procedural skill. In 

t » C ^* n "* (J982). Tutoring rules for guiding a case method dialogue. 

Acao^r^.^^fo^' adUOd * °' S, ° iJ * J ' S " B — ■ ^ 

ft. B. Davis, £. Joekusch A C. McKnlght (1978). Cognitive Processes In 
Learning Algebra. JflwuiAl III cminrpw'n HathnmaHrai C ^h^jnnr , 2t ppi-320. 

York:"'van Nos trend. ( IM,) ' CiLlUuQl4 Af Uhmfftlr . ts ^lUa LtLAxoiiLg >r»»» Ntw 

. -i IZ' 6 jldste1n (1982). The genetic graph: a representation for the 

i a uem.. I j P J°"J1!« k T U ; 9 *: f I ^^0*^ imaeiM SyjJLM*. edited by 
o. sieeman & J.S. Drown. New York: Academic Press, pp 61-77. 

tn. HaP V' y u * u 0, t!; slMm * n < 1973 >' Towards Intelligent Teaching 
Systems, lnl* J* Man-H/irhipg St udHai. ft, pp215-2Z6. 

, c °' J Ku6 = h ; ma " n ("81). Algebra in UlilflXJUCi UnrinrttanHlnn HetnenmUfl. . 

iiilfi. edited by K.H. Hart. London: John Murray. ppl02-119. " 

S" l Vl % l 19fl0) - fillJl 1 0f Knowled 9e *" Algebra. Paper presented to the 
AtRA Annual Conference, 1900. 

, . M * l « (J 98 *) Towards a procoss model Tor high school algebra errors. In 

LLLfllliMflJL T u toring Syalflna. edited by 0. Sieeman and J.S. Brown. Academic 
Press, pp 25-60. 

E.R. Michenor (1978). Understanding Understanding Mathematics. ConiLLliiifi 
Siliauu 1. pp 361-383. 

O.A^ Norman (1981). Cate(jor izatlon of Action slips. Psvchoinnic.il 
fl-av law, aa. ppt-15. * " — 

« 

u - V'. , Pa1ge J H,A> Siin °n (1966) Cognitive Processes in Solving Algebra 

L i *!!;,* ! SQlv1fl 0 ; HfliMfnn, Matted Ann Thnory. edited by b. 

Kiemmuntz, Published by Wiley. — — t 

*,tif?'?\ ft*???* M,J> Smith (1901). Modelling pupil's Problem Solving. 
Art 1 f tcifll IniaiULgAnfifi, pp 171-187. * 



9 

ERIC 



27 

BEST COPY 



Page 27 

T ?' 0M2) Assessing compotenco In basic Algebra. In InifiiLUtnl 

Iujajlja SjtAiiflA. edltad by O. Sloeman and J.S. B?own. AcadamJclrtisTTj 

edited by 0. Sleeman and J.S. Brown, Academic Prass, pp 1-12. y "" M ' 

J? Sr«2 P U S2 < S!a X ! *A"iaa amana. edited by 0. Sleemin J 

J. 5. drown, liaw York: Academic Prees. pp 99-na, 

• n ,r n ?„J' 5 1,em J? i lfl ^? ) * , A Ru1a r)irected Modelling system. In HaOilia 
pfJSfSp 4M-5to! Carbonall. and T. H, Mitchell, (editors)?^ 

MffowTm 1 !^ a^Ei! 0 und9rsur,d »■»«'•' of 

• * J' 11 ',, S,, *? an . (*•««)• flas^ Algebra revisited: a stud* with 

hj,«j. v °r- jtvm. -an;. rT^^^ 

instructional s&lmei tbch, » aB ort. cis-n . LPBniL1VB ^ 

r5: )! anL,hn n?««). 0« the Raprasantatlon of Procedures In Repair Theory, 

u.u , K :, VanL, !? n ("83b). Felicity conditions for human skill acquisition: 
Validating an Al-basad theory. mQl £AB£ t ech. cis-gy a ^ u,s, " on - 

SsiJL I? Un p fl p t63 T :t7?; She8 (l981) ' Err ° r8 10 Ch ""«" , « ^traction, toga^ 
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RULE NAME 



FIN2 

SOLVE 

SIMPLIFY 

AOOSUB 

MULT 

XADDSUB 

NTORHS 

REARRANGE 

XTOLHS 

BRA1 

BRA2 



EigUXfi 1 

«) RULES *or the ALGEORA domain (slightly stylized) 



CONDITION 

(X - M/N) 

(M • X • N) 

(X • N/N) 

(<h$ N N rhs) 

(lhs M • N rhs) 

(lhs N«X ♦ !- N«X rhs) 

(Ins m » phs) 

(lhs +1-H N*X phs) 

(lhs • *l- M«X rhs) 

(lhs < N > phs) 

(lhs M««N«X *I-P> phs) 



ACTION 

((M N) ) OP ((M)) 

(X • H/M) op (INFINITY) 

(X - M VN* ) 

(lhs [evaluated] Phs) 

(lhs [evaluated] rhs) 

(lhs (M ♦ !- N) • X phs) 

(lhs ■ phs -I* M) 

(lhs *l- N«X M phs) 

(lhs -!♦ M«x ■ phs) 

(lhs N phs) 

(lhs M«N*X M*P phs) 



Where M. N and P apt Integers, lhs & phs ape genepal patterns (which may be null), ♦!- 

means either ♦ op - may occup. and < and > represent standard algebraic brackets. 

for eaample. the MULT rule changes the state (3 • X • 6 ♦ 3 • 4) to (3 • X » 6 ♦ 12) where 

the variables in the pattern, lhs, rhs. M and N. are bound to (3 • X - 6 ♦). null. 3 and 4 

respectively. 

See Figure lc for a complete trace of this and several other tasks. 



b) Same. HAL-RULES torn lh& Dflaain 



RULE NAME 



CONDITIO* 



ACTION 



MSOLVE 

MNTORHS 

H2 NTORHS 

M3NT0RHS 

MXTOLHS 

M1BRA2 

M2BRA2 



(M»X - N) 
(lhs ♦!- M • rhs) 
(lhsl M lhs2 - rhs) 
(lhsl ♦!- M lhs2 ■ rhs) 
(lhs - ♦!> M«X rhs) 
(lhs M «<N«X P» rhs) 
(lhs M«< N*X P> rhs] 



(X - M/N) or ( INFINITY) 

(lhs ■ rhs H- M) 

(lhsl Ihs2 ■ rhs -I* M) 

(lhsl ♦ Ihs2 ■ rhs ♦ !- M) 

(lhs ♦!- N«X - rhs) 

(lhs H*H*t ♦ !- P rhs) 

(lhs M«N«X M ♦ !- P rhs) 



Using the same conventions as above. 



e) P a ir s at correct and "buggy: modals. «»»euHnp t ypicm tftiki . 

1) shows the correct model (MULT A0OSU8 SOLVE FINS) 
and the "buggy" model 

(ADOSUB MULT SOLVE FIN2) solving 3 • X - 5 + 3 • 4. 

[The first lino gives the Initial state and all subsequent lines give 
the rule which fires And the resulting state.] 



3*X"5*3*4 3*X«6*3*4 

MULT 3 • X ■ 6 ♦ 12 AODSUB 3 • X - 8 • 4 

ADQSUB 3 • X - 17 MULT 3 ■ X ■ 32 

SOLVE X - 17/3 SOLVE X - 32/3 

FIN2 (17 3) FIN2 (32 3) 



ERIC 
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11) shows the correct modol (NTORHS AOOSUB S0LV£ FIM2J 
end the "buggy" r.ode1 ' 

(MN TORUS AOOSUB SOLVE FIU2) solving 4 • X * 6 - 19. 



4 • X ♦ 8 ■ 19 

NTORHS 4 • X • 19 - 6 
AOOSUB 4 • X ■ 13 
SOLVE X - 13/4 
FXN2 (13 4) 



4 • X ♦ 6 - 19 

MNTORHS 4 • X • 19 ♦ 8 
AOOSUB 4 • X • 25 
SOLVE X - 26/4 
FIN2 (26 4) 



f i g ura 2 Typic a l luks tat i&&h lash-sal &ajt which olLiUI aca oaina facuuad on 
Task-set Rules Focussed On Typical Task 



2 


SOLVE 


3 


AOOSUB 


4 


MULT 


6 


XAODSUB 


6 


NTORHS 


7 


REARRANGE 


8 


XTOLHS 


9 


BRA1 


10 


8RA2 


11 


AOOSUB&NULT 


12 


ADOSUB&XAOOSUB 


13 


AD0SUB&BRA2 


14 


MULTtXADDSUB 


15 


HULT&BRA2 



5 • X i 
• 4 • 

4 • X ■ 



7 

8*3 

2 • 2 

3 • X • 10 

4 • 18 
X « 18 

2 • X ♦ 3 
6 • <3 ♦ 1> 
4 • <2 • X ♦ 3> 
2*4*8 
X ♦ 4 • X ■ 16 
1 2 ♦ 4 • <2 • X 
X ♦ 2 • X ■ 12 
2 • 3 • <2 • X 



♦ 3> 

♦ 3> 
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p lM ? from " h '? h mal r « 1 «» "ore Inferred. (Mota the teacher speciflod the way In 
which the X-coeff ident should be represented. MIL iao ihai MJDtt Ql SM uttaJH 4 u 



Task-set 6 



Task is (2 • X ♦ 4 • X • 12) Student's solution was (1 • X - 3) 

Task Is 2 • X ♦ 3 • x - 10) Student's solution was (2 • X - 10 - 2 - 3) 

Task Is (3 • X ♦ 2 • X - 11) Student's solution was (1 • X « 6 // 2) 

Task t 2 • X ♦ 6 • X - 10) Student's solution was (1 • X - 1) 

Task is (3 • X ♦ 4 • X • 9) Student's solution was (1 • x • 1) 

Task a 2 • X ♦ 4 • X - 3) Student's solution was (1 • X - -3 // 2) 

Task is (4 • X ♦ 2 • X - 4) Student's solution was (8 • X - 4) 

M gu rf It. Protocol apparently showing M • X -> H ♦ X (student AB17), 
Tesk-set 6 

Task is (2 • X ♦ 4 ■ 16) Student's solution was (1 • X - 8 // 3) 
Task Is (2 • X ♦ 3 ■ 0) Student's solution was (1 • X - g // 6) 
Task is (3 • X - % • 6) Student's solution wes (1 • X - -6) 
Task is (2 • a ♦ 8 - 10) Student's solution was (1 • X - 10 // 7) 
Task is (6 * X ♦ 4 - 0) Student's solution wss (1 • X ■ 3 // 8) 
Task is (5 • X ♦ 2 ■ 6) Student's solution was (1 • X - 6 // 7) 

Eigutl an. Protocol apparently showing M • X ♦ H •> (M ♦ N) • X (student AB20). 



Task-set 7 



Task Is (4 ♦ 2 • X - 16) Student's .solution was (1 • X • 8) 
Task is (2 ♦ 4 • X - 14) Student's solution was (1 • X - 6) 
Task is (3 ♦ 8 • X - 11) Student's solution was (1 • X ■ -4) 
Task is (4 ♦ 6 • X - 11) Student's solution was (1 • X • -13) 
Task Is (4 ♦ 6 • X • 6) Student's solution wes (1 • X • -14) 
Task is (6 ♦ 2 • X ■ 8) Student's solution was (1 • X • -2) 

Ugut* It. Protocol apparently showing M*N»x->M«N*X (student AB3). 
Tesk-set 8 

Task is (4 • X • 2 • X ♦ 6) Student's solution was (1 • X • 6) 
Task Is (3 • X ■ 2 • X ♦ 6) Student's solution was (1 • X - 6) 
Task is (3 • X • -2 • X + 7) Student's solution was (1 • X • 4) 
Task is (4 • X ■ 2 • X ♦ 3) Student's solution was (1 • X ■ 0 // 2) 
Task is (4 • X • -2 • X ♦ 8) Student's solution was (1 • X • 6) 
Task 1a (6 • X ■ 2 •• X ♦ 3) Student's solution was (1 • X ■ 11 // 2) 

UiU£A Id. Protocol apparently showing MX-NX*P->X*x-M+M*P (student Afll), 
Task-set 7 

Task 1s (4 ♦ 2 * X ■ 16) Student's solution was (1 • X - 2) 
Task 1s (2 *■ 4 • x - 14) Student's solution was (1 • X - 4 // 2) 
Task 1s (3 ♦ 6 • X - 11) Student's solution was (1 • X • 5 // 3) 
Task is (4 ♦ 6 • X • 11) Student's solution was (1 • X - 6 // 4) 
Task is (4 ♦ 6 • X • 6) Student's solution was (1 • X - -5 // 4) 
Task is (6 ♦ 2 • X - 6) Student's solution was (1 • X • 8 // 2) 

Mgura 2a. Somewhat erratic protocol: 3 responses conform to 
H U M ■ P ■> H • » • II (Student AB7) . 
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mal-rule & represents ^ common factor*. 

It. NMtIM -> M • X • H 

-> M • X ♦ X 

•> M ♦ X ♦ N ♦ X 

tb. M • X ■> M ♦ X 

2. N ♦ N • X • -> M • N ♦ X ■ 

•> M ♦ N ♦ X • 

3- M * X ♦ M ■ -> H ♦ X ♦ II ■ 

•> M • X • H ■ 
■> (M ♦ N) • X ■ 

4. M • X ■ N«P ■> X ■ N 

6. N • X • N • X ♦ P -> X*X-M*N*P 

8. M • X - N ♦ P ■> N * X ■ N 

-> N • X • P 

M • X - N •> X - N 

M • X ■ N -> X ■ (N/F)/M 

-> X ■ N/(N/F) 

g. N • <N • X ♦ P> -> H • X ♦ M • P 



F 1 aur!. tJ, ?I Ji IV 1 £2 Ul d hav \ be ! n *P«c1f1ed 1n exactly the same format as that used In 
Flgurai la&lb; tha current form his been used for brevity. However. In that earlier 
notation rules lb and 9 above would become: aarner 

CONDITION ACTION 

(lhs M • X rhS) (ihs M ♦ X rhs) 

(lhs M • < N • X ♦ P> rhs) (lhs M • X ♦ M • P rhs) 

•here lhs & rhs ara general patterns. 
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Protocol for a student who has a number of "Alternative Methods". 
Studtnt AB17 oft task-sat 0. 

a) The task given was: 2 • X ♦ 3 • g 

Student writes 1) 2X • 0 - 3 

2) X ■ 3 

i 

Interviewer writes X ■ 9 • 3 ♦ 2 

Interviewer: Could you say whether your step 1) above or what I've Just written is 
cor r#c \ * 

Student says he really eould not. 

b) The task given was: 2 • X ♦ 4 • ifl 

Student writes 1) 2X - 16 - 4 

2) 2X • 12 

3) X - 0 

Interviewer writes X ■ IS - 4 - 2 

Interviewer: Could you say whether your stop 1) above or what I've just written is 
cor r 6 c w • 

Studtnt says his 1) probably Is. 
Inttrvltwer: Can you say why? 
Student: I'm afraid not. 

Student: That's right, It would. 

Interviewer: Which of these do you thiik Is correct? 

Student: Really not sure. I often havo a lot of methods to choose between, which 
makes it pretty confusing. I sometimes have as many as 6 or 6. D ""«. 

conver " t,0ft continues. After this point the student voluntarily offers 2 
or 3 solutions to each task, as In the next task.] 1 

c) The task given was 4»X-2*X*6 

Student writes 1)X»2*4*G 

X - 4 

Then suggests the following reworking: 

1) 4X ■ 2X ♦ 0 

2) 4X - ex 
Then Quits. 

Interviewer: Which solution do you think 1s rlghtT^ 
Student: Oh. I'm not really sure. 

Interviewer: If you ware a betting man, which would you put your money on7 
Student: Probably the first. 
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H gure ft 

Three examples of very consistently used MAL-RULCS. 



I) Student ABU on task-sat 7. 
a) The task given eat: 4 ♦ 2 • x - 16 

Student writes 1) OX ■ 10 

2) X - 2.8606 



b) The task given was: 2 ♦ 4 • X ■ 14 

Student writes l) 6 • X ■ 14 

2) X - 2.333 



c) The task given was: 3 •: 6 • X • U 

Student writes 1) 8 • X • 11 
(and Is told she can leave 1t In that form) 

d) The task given was: 6 - 3 • X • 11 

Student writes 1) 2 • X ■ 11 
(and Is told she can leave It 1n that form) 



II) Student ABI7 on task-set 0 

a) The task given was: 2 • X ♦ 4 • X - 12 

Student writes ' l)°X*X>i2-2-4 

2) X •• 2 ■ 0 

3) X - ROOT 0 

b) The task given was: 2 • X ♦ 3 « X ■ 10 

Student writes l)*X*x-10-2-3 

2) X •• 2 - 6 
(and Is told he can leave H 1n that form) 

c) The task given was: 2 • X - 3 • X • 10 

Student writes 1)*X«X-io-2*3 

2) X •• 2 - u 
(and Is told ho can leave It In that form} 



III) Student AP18 on task-sets 5, 6, 7 and 8. 

a) The task glvon was: 2 • X ♦ 3 • x • 10 

Student writes 1) 2 • X ■ 10-2-3 

2) 2 • X ■ 8 

3) X - 2.6 

b) Tho task given was: 3*X+6*X>24 

Student writes 1) X ♦ X • 24-3-5 

2) 2 • X - 16 

3) X - 8 

c) The task given was: 3*X+4*x+S*x-24 

Student writes 1) X ♦ x + X • 24 -3-4-6 

2) 3 ■ X ■ 12 

3) X - 4 

d) The task given was: 2 • X ♦ 4 • 20 

Student writes 1) X « 20 - 2 - 4 

2) X - 14 
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e) lhe task given was: 3 • x ♦ S • 7 

Student orUit 1) X • 7 - 3 - 5 

2) X - -1 

f) The task given his: 4 ♦ 3 • X • V, 

, Student writes t) X • 14 - 3 • 4 

2) X - 7 

g) The task given was: 6 ♦ 6 • x • 20 

Student writes 1) X ■ 20 - 6 - 6 

2) X • 0 

h) The task given was: *»X>2«x»6 

Student writes 1)2«X- -4*2*6 

2) 2 • X ■ 4 

3) X • 2 

Student then wrote l)x-x-2*6-4 

2), 0 - 4 
end QUITS. 

1) The task given was: 6 • X • 3 • X ♦ ft 

Student writes 1) 0 ■ 4 

and QUITS. 
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Tho cornet put troo for tht oquatloa 6«X«3»x + 12 




A "two-lw.1- rooroMnutlon for tho ««, «,, M tien vh.ro i !. 
folloviog Buady IMi, r.pr...»„ . -plu. bo,". th.t i.. M th . 

•ntitt.. «ta .h. b., „. ep . mtd on k , tha , Mltlon 
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